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Abstract. A rectangulation is a tiling of a rectangle by a finite number of 
rectangles. The rectangulation is called generic if no four of its rectangles share 
a single corner. We initiate the enumeration of generic rectangulations up to 
combinatorial equivalence by establishing an explicit bijection between generic 
rectangulations and a set of permutations defined by a pattern-avoidance con- 
dition analogous to the definition of the twisted Baxter permutations. 



The main characters in this paper are tilings of a rectangle by finitely many 
rectangles. A cross in such a tiling is a point which is a corner of four distinct tiles. 
Fixing a rectangle S and considering the space of all tilings of S by n rectangles, 
with a uniform probability measure, the set of tilings having one or more crosses 
has measure zero. Thus we call a tiling generic if it has no crosses. 

We consider generic tilings up to the natural combinatorial equivalence relation 
which we now describe. We orient S so that its edges are vertical and horizontal. 
A rectangle U in a tiling R is below a rectangle V if the top edge of U intersects the 
bottom edge of V (necessarily in a line segment rather than in a point). Similarly, 
U is left of V if the right edge of U intersects the left edge of V. A tiling R of a 
rectangle S is combinatorially equivalent to a tiling R' of a rectangle S' if there is a 
bijection from the rectangles of R to the rectangles of R' that exactly preserves the 
relations "below" and "left of." A generic rectangulation is the equivalence class 
of a generic tiling. We will often blur the distinction between generic rectangula- 
tions (i.e. equivalence classes) and equivalence class representatives, in particular 
specifying an equivalence class by describing a specific tiling. 
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Table 1. The number of generic rectangulations with n rectangles 



Our main result is a bijection between generic rectangulations with n rectangles 
and a class of permutations in S n that we call 2-clumped permutations. These are 
the permutations that avoid the patterns 3-51-2-4, 3-51-4-2, 2-4-51-3, and 4-2-51-3, 
in the notation of Babson and Steingn'msson \~5\ , which is explained in Section El 
The author's counts of generic rectangulations, for small n, are shown in Table [l] 

We define fc-clumped permutations in Section[2j For now, to place the 2-clumped 
permutations in context, we note that the 1-clumped permutations are the twisted 
Baxter permutations, which are in bijection with the better-known Baxter per- 
mutations. Baxter permutations are also relevant to the combinatorics of rectan- 
gulations. Indeed, Baxter permutations are in bijection [21 119j with the mosaic 
floorplans considered in the VLSI (Very Large Scale Integration) circuit design 
literature [13] • Mosaic floorplans are certain equivalence classes of generic rectan- 
gulations. (A similar result linking equivalence classes of generic rectangulations to 
pattern-avoiding permutations is given in 0].) In light of results of pQ, the bijection 
from Baxter permutations to mosaic floorplans can be rephrased as a bijection to a 
subclass of the generic rectangulations that we call diagonal rectangulations, which 
figure prominently in this paper. 

The symbol G n will denote the set of 2-clumped permutations. Let gRec„ be 
the set of generic rectangulations with n rectangles. The bijection from G n to 
gRec„ is defined as the restriction of a map j : S n — > gRec„. We show that 7 
is surjective and that its fibers are the congruence classes of a lattice congruence 
on the weak order on S n . We do not prove directly that the fibers of 7 define a 
congruence. Instead, we recognize the fibers as the classes of a congruence arising 
as one case of a construction from |17j , where lattice congruences on the weak order 
are used to construct sub Hopf algebras of the Malvenuto-Reutenauer Hopf algebra 
of permutations. The results of [17] show that the 2-clumped permutations are a 
set of congruence class representatives. Thus the restriction of 7 is a bijection from 
G n to gRec n . 
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Note added in proof. After this paper was accepted, the author became aware of 
a substantial literature studying generic rectangulations under the name rectangular 
drawings. This literature includes some results on asymptotic enumeration as well 
as computations of the exact cardinality of gRec n for many values of n. See, for 
example, [3l 1111 H4] . In particular, the main result of this paper answers an open 
question posed in [31 Section 5]. 

2. Clumped permutations 

In this section, we define fc-clumped permutations. We begin with a review of 
generalized pattern avoidance in the sense of Babson and Steingrimsson [5 . Let 
V = Vi • • • Vk G Sk, and let y be a word created by inserting a dash between some 
letters of y\ ■ ■ ■ yk, with at most one dash between each adjacent pair. A subsequence 
x i± ■ ■ ■ Xi k of X\ - ■ ■ x n is an occurrence of the pattern y in a permutation x £ S n 
if the following two conditions are satisfied: First, for all j, I £ [k] with j < I, the 
inequality Xy < x^ holds if and only if yi < yi holds. Second, if yj and yj+i are 
not separated by a dash in y, then ij = — 1. That is, the dashes indicate which 
elements of the subsequence are not required to be adjacent in x. For example, 
the subsequence 4512 of 45312 £ S$ is an occurrence of the pattern 3-4-1-2, or an 
occurrence of the pattern 34-12, but not an occurrence of the pattern 3-41-2. If 
there is no occurrence of the pattern y in x, then we say that x avoids y. 

To define fc-clumped permutations, we first consider the twisted Baxter permuta- 
tions, defined in [17] and shown in unpublished notes by West |18j to be in bijection 
with Baxter permutations. A published proof can be found in [15] or [12] . The 
twisted Baxter permutations are the permutations that avoid the patterns 2-41-3 
and 3-41-2. This pattern-avoidance condition on a permutation x — X\ ' ■ ■ x n can 
be rephrased as follows: For every descent Xi > Xi+i, the values strictly between 
Xi+i and Xi are either all to the left of X4 or all to the right of Xi+\. (The Bax- 
ter permutations are defined by a similar condition: They are the permutations 
avoiding 3-14-2 and 2-41-3.) 

In any permutation x, we define a clump associated to a descent Xi > Xi+i to be 
a nonempty maximal sequence of consecutive values strictly between Xi and Xi+\, 
all of which are on the same side of the entries XiXi+\. No requirement is made on 
the positions, relative to each other, of the values in the clump. For example, in 
the permutation 269153847 £ Sg, there are four clumps associated to the descent 
9 > 1, namely 2, 345, 6, and 78. 

The pattern avoidance condition defining twisted Baxter permutations is that 
each descent Xi > has at most one clump, so we refer to twisted Baxter per- 
mutations as 1-clumped permutations. More generally, a k-clumped permutation 
is a permutation x such that each descent Xi > X4+1 has at most k clumps. One 
can easily rephrase the definition of fc-clumped permutations in terms of general- 
ized patterns avoidance (avoiding 2(|)!(| + l)! generalized patterns if fc is even or 
2 (2/' (^2^)' generalized patterns if k is odd). For example, the 2-clumped per- 
mutations, which play the central role in this paper, are the permutations avoiding 
3-51-2-4, 3-51-4-2, 2-4-51-3, and 4-2-51-3. By convention, the only (-l)-clumped 
permutation is the identity. The 0-clumped permutations are the permutations 
such that if Xi > Xi+i then Xi — 1 = aJj+i,. Equivalently, they are the permutations 
avoiding 31-2 and 2-31. These permutations in S n are in bijection with subsets of 
{1,2, •■•71— 1}. The 3-clumped permutations appear not to have been considered 
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before. For n from 1 to 9, the numbers of 3-clumped permutations are 1, 2, 6, 24, 
120, 712, 4804, 35676 and 284816. 

The weak order on Sk is a lattice whose cover relations are x<y with x = x\ • • • xj. 
and y = yi ■ ■ ■ Uk such that Xi = yi+i < y% — a;<+i for some i £ [fc — 1], with Xj = y,j 
for j ^ {i, i + 1}. A join-irreducible permutation is a permutation x £ Sk with 
exactly one descent, meaning that, for some index i £ [fc — 1], we have Xj > a^j+i 
but < for every j £ [fc — 1] with j ^ i. (Such a permutation is join- 

irreducible in the weak order in the usual lattice-theoretic sense.) 

We now review a construction from |17l Section 9]. A join-irreducible element 
x £ Sk is called untranslated if its unique descent Xi > a^+i has Xi = k and 
Xi + i = 1. In this case, a scramble of x is any permutation y such that yi = fc, 
yi+i = 1 and every entry j with 1 < j < fc occurs to the left of position i in x if and 
only if it occurs to the left of position i in y. Let y be a scramble of x and let y be 
obtained from y by inserting a dash between each pair of consecutive entries except 
between fc and 1. We say that the scramble y of x occurs with adjacent cliff if the 
pattern y occurs. 

Let C be any collection of untranslated join- irreducible elements in Sk, with fc 
varying, so that, for example, C may be {312, 2413}. The following is essentially 
Q21 Theorem 9.3]. 

Theorem 2.1. For each n, there exists a unique congruence H(C) n on the weak 
order on S n with the following properties: 

(i) A permutation z is the minimal element in its %{C) n -class if and only 
if, for every x £ C and all scrambles y of x, the permutation z avoids 
occurrences of y with adjacent cliff. 

(ii) Suppose w < z in the weak order, and let Zi and Zi + \ be the adjacent entries 
of z that are swapped to convert z to w, with Z\ > Zi+i- Then w = z 
modulo H(C) n if and only there exists x £ C, a scramble y £ Sk of x, and 
an occurrence of y in z such that the entry of z corresponding to the entry 
k in y is Zi and the entry of z corresponding to 1 in y is Zi+\. 

In [T7], the congruence H(C) n is constructed for the purpose of building combi- 
natorial Hopf algebras. Here, we can take Theorem |2.1| as the definition of H(C) n . 



Property (i) in Theorem 2.1 is a direct restatement of jTTJ Theorem 9.3], while 
property (ii) is the key point in the proof of (17L Theorem 9.3]. It is easy and well- 
known that in a congruence on a finite lattice, each congruence class is an interval. 
Thus a congruence is uniquely determined by the set of cover relations w < z such 
that w = z. Furthermore, the minimal permutations described in Property (i) are 
a system of congruence class representatives. 



Let r be the congruence "H({35124, 24513})„ on S n . Theorem 2.1 specializes to 
the following: 

Proposition 2.2. 

(1) A permutation is the minimal element in its T-class if and only if it is a 
2-clumped permutation. 

(2) Suppose x < y in the weak order, and let e and a be the adjacent entries 
that are swapped to convert y to x, with a < e. Then x = y modulo T if 
and only if there are entries b, c, and d in y with a<b<c<d<e such 
that b and d are on the same side of ea, while c is on the other side of ea. 
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More generally, for each k > — 1, there is a congruence described by Theorem |2.1| 
such that the minimal elements of congruence classes are exactly the fc-clumped 
permutations. 

3. The map from permutations to generic rectangulations 

In this section, we define a map 7 from S n to gRec n . We will see, in Section [4] 
that 7 restricts to a bijection from the set of 2-clumped permutations to gRec„. 
The key point in the proof that 7 restricts to a bijection will be the fact that its 
fibers are the congruence classes of the congruence T defined at the end of Section [2] 

To define the map 7, we first consider a smaller class of rectangulations which we 
call diagonal rectangulations and a map from permutations to diagonal rectangula- 
tions. The diagonal of the underlying rectangle S is the line segment connecting the 
top-left corner of S to the bottom-right corner of S. Recall that each rectangulation 
is a combinatorial equivalence class. A rectangulation is a diagonal rectangulation 
if it has a representative in which each rectangle's interior intersects the diagonal. 
A diagonal rectangulation is in particular a generic rectangulation, because if any 
four rectangles have a common vertex, it is impossible for all of their interiors to 
intersect the diagonal. Diagonal rectangulations have been considered under other 
names, for example in (TJ |SJ [TU] . 

We now review, from |15j . the definition of a map p from permutations to di- 
agonal rectangulations. Maps closely related to p have appeared prior to [TS], for 
example in [TJ [10] . To define p, first draw n + 1 distinct diagonal points on the di- 
agonal of S, with one of the points being the top-left corner of S and another being 
the bottom-right corner of S. Number the spaces between the diagonal points as 
1,2, ... ,n, from top-left to bottom-right. Given x G S n , read the sequence x\ ■ ■ ■ x n 
from left to right and draw a rectangle for each entry according to the following 
recursive procedure: 

Let T be the union of the left and bottom edges of S with the rectangles drawn 
in the first i — 1 steps of the construction. It will be apparent by induction that 
T is left- and bottom-justified. To draw the i th rectangle, consider the label Xi 
on the diagonal. If the diagonal point p immediately above/left of the label Xi is 
not in T, then the top-left corner of the new rectangle is the rightmost point of T 
that is directly left of p. If p is in T (necessarily on the boundary of T) , then the 
top-left corner of the new rectangle is the highest point of T directly above p. If 
the diagonal point p' immediately below/right of the label Xi is not in T, then the 
bottom-right corner of the new rectangle is the highest point of T that is directly 
below p' . If p' is in T then the bottom- right corner of the new rectangle is the 
rightmost point of T that is directly to the right of p' . 

Example 3.1. Figure [T] illustrates the map p. In each step, the new rectangle is 
shown in red (the darker gray when not viewed in color), and the set T consists of 
the white rectangles together with the left and bottom edges of S. The part of S 
not covered by rectangles is shaded in light gray. 

Given a diagonal rectangulation R, we number the rectangles in R according 
to the position of their intersections with the diagonal, starting with rectangle 1, 
which contains the top-left corner of S and ending at rectangle n, which contains 
the bottom-right corner of S. Thus, for example, in constructing the rectangulation 
p[x), we first construct the rectangle numbered 27, then the rectangle numbered x^, 
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FIGURE 1. Steps in the construction of p{8@75©2@6@9@314@) 
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Figure 2. Wall slides 



etc. Say a permutation X — 33i * * * IS compatible with a diagonal rectangulation R 
if and only if, for every i € [n] , the left and bottom edges of the rectangle numbered 
Xi are contained in the union of the left and bottom edges of S with the rectangles 
numbered xi,... , £»_ 1. Equivalently, x is compatible with R if, for every i G [n], 
the union of the rectangles numbered x±,...,Xi is left- and bottom-justified. The 
following fact is established in the proof [15l Proposition 6.2], which asserts that p 
is surjective. 

Proposition 3.2. Given a diagonal rectangulation R, the fiber p^ 1 (R) is the set 
of permutations in S n that are compatible with R. 

The following proposition, which is the concatenation of [TS1 Proposition 4.5] and 
[TBI Theorem 6.3], shows in particular that the fibers of p constitute a congruence 
of the kind described in Theorem 12. II 



Proposition 3.3. Suppose x <y in the weak order, and let d and a be the adjacent 
entries that are swapped to convert y to x, with a < d. Then p(x) = p(y) if and 
only if there are entries b and c, with a < b < c < d, such that b and c are on 
opposite sides of da in y. 

In some of the literature on floorplanning for integrated circuits, generic rect- 
angulations are referred to as mosaic floorplans, but in that literature, the term 
mosaic floorplan always implies a coarser equivalence relation than the combinato- 
rial equivalence used to define rectangulations as equivalence classes. Specifically, 
two generic rectangulations are equivalent as mosaic floorplans if and only if they 
are related by a sequence of what we call wall slides. A wall in a rectangulation R 
is a line segment in the underlying rectangle S, not contained in an edge of S, that 
is maximal with respect to the property of not intersecting the interior of any rec- 
tangle of R. A wall slide along a wall W is the operation taking two walls of R that 
end in W, from opposite sides, and sliding them past each other, without changing 
any of the other incidences in R. Wall slides come in two orientations, as illustrated 
in Figure [2j The following is a very special case of [TJ Theorem 4] . 

Proposition 3.4. Given a generic rectangulation R, there exists a unique diagonal 
rectangulation R' such that R and R' are equivalent as mosaic floorplans. 



To see Proposition 3.4 as a special case of p] Theorem 4], we need the definition 
of a diagonal rectangulation given in [15, Section 5] : Let X be a set of n — 1 distinct 
points on the diagonal of S, none of which is the top-left corner or bottom-right 
corner of S. Then a diagonal rectangulation of (5*, X) is a generic rectangulation 
such that every wall contains a point of X and such that every point of X lies on a 
wall. By |15[ Proposition 5.2], this definition is equivalent to the earlier definition. 

Suppose R is a generic rectangulation and let R' be the diagonal rectangulation 
that is equivalent to R as a mosaic floorplan. As before, number the rectangles in 
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Figure 3. A generic rectangulation 



R' according to the position of their intersections with the diagonal, 1 to n from 
top- left to bottom-right. Letting this numbering propagate along wall slides in the 
obvious way, we obtain a numbering of the rectangles of R. For each vertical wall 
W of R, we produce a permutation aw of a subset of [n] as follows: Moving from 
the bottom endpoint of W to the top endpoint of W, when we come to a wall 
W' that is incident to W on the left, we record the number of the rectangle that 
has its right edge in W and its bottom edge in W'. When we come to a wall W 
that is incident to W on the right, we record the number of the rectangle that 
has its left edge in W and its top edge in W . The resulting partial permutation 
o~w is called the wall shuffle associated to W , because it is obtained by shuffling 
two sequences: the decreasing sequence of numbers of rectangles whose right edge 
is contained in W (excluding the bottom such rectangle) from bottom to top and 
the decreasing sequence of numbers of rectangles whose left edge is contained in W 
(excluding the top such rectangle) from bottom to top. 

For each horizontal wall W, we construct the wall shuffle associated to W in 
a similar manner. Moving from the left endpoint of W to the right endpoint of 
W, when we come to a wall W that is incident to W on the top, we record the 
number of the rectangle that has its bottom edge in W and its right edge in W . 
When we come to a wall W that is incident to W on the bottom, we record 
the number of the rectangle that has its top edge in W and its left edge in W . 
The partial permutation aw, in this case, is obtained by shuffling two increasing 
sequences: the sequence of numbers of rectangles whose bottom edge is contained 
in W (excluding the rightmost such rectangle) from left to right and the sequence 
of numbers of rectangles whose top edge is contained in W (excluding the leftmost 
such rectangle) from left to right. 

Example 3.5. Figure|3]shows a generic rectangulation R whose associated diagonal 
rectangulation R' is the rectangulation from Figure [l] The numbering of rectangles 
is inherited from R' . Tables [2] and H] show the wall shuffles associated to R. 

Specifying a generic representation R is equivalent to specifying the associated 
diagonal rectangulation R' along with the wall shuffles for each wall. For some 
walls, there may be only one shuffle possible, and this unique shuffle may be empty. 
The shuffles may be chosen arbitrarily (among shuffles of the appropriate rectangle 
numbers) and independently for each wall, and each sequence of choices of R' and 
the wall shuffles yields a different generic rectangulation. 
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Table 2 . Wall shuffles in vertical walls of the rectangulation of Figure 3 



Rectangles above wall 


Rectangles below wall 


Wall shuffle 


1 


2,3 
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2, 3,4 


5, 6, 9, 10 
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5, 6 
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13, 14, 15 


@@@ 



Table 3. Wall shuffles in horizontal walls of the rectangulation of Figure 3 



When a wall slide is performed along a wall W, the move alters aw by swapping 
two adjacent entries which number rectangles on opposite sides of W. Since a wall 
slide only changes the combinatorics locally, performing a wall slide along W does 
not alter the wall shuffle for any other wall. 

We now define the map 7 : S n — > gRec n . Let x = x\X2 • ■ • x n € S n and construct 
R! = p(x). Let W be a vertical wall in R! and consider the rectangles in R' 
having their right edges contained in W. By construction, the numbers of these 
rectangles form a decreasing subsequence of X1X2 ■ ■ ■ x n . Similarly, the numbers 
of the rectangles in R' having their left edges contained in W are a decreasing 
subsequence of x\X2 ■ ■ ■ x n . Thus we can specify a wall shuffle aw by taking the 
subsequence of x\x<i • • • x n consisting of the appropriate rectangle numbers. For a 
horizontal wall W, the numbers of the rectangles having their top edges contained in 
W form an increasing subsequence of X1X2 ■ • • x n and the numbers of the rectangles 
having their top edges contained in W form an increasing subsequence of X1X2 • • • x n , 
so, in this case as well, we can specify a wall shuffle for W by taking an appropriate 
subsequence of X1X2 • • • x n . The diagonal rectangulation R' together with all of 
these wall shuffles define the generic rectangulation 7(2;). 

Example 3.6. This is a continuation of Examples |3.1| and |3.5| Figure [T] shows the 
construction of p(x) for x = 8@75@2@6©9@)314(i2). To construct 7(2;), we look 
at each wall of p(x). For example, p(x) has a horizontal wall W with rectangles 
2, 3, and 4 above W and rectangles 5, 6, 9, and 10 below W. The restriction of x 
to the set {2,3,6,9,10} is 269 © 3. Thus 7(2;) is a rectangulation that is mosaic 
equivalent to p(x) and that has a wall shuffle 269 ©3. Considering similarly the 
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other five horizontal walls of p(x) and the eight vertical walls of p(x), we see that 
7(0;) is the rectangulation shown in Figure [3] (Cf. Tables [2] and [3j) 

Example 3.7. Figure [4] shows the map 7 applied to every permutation in S4. 
The permutations in 64 are shown in the weak order, and the 24 rect angulations 
in gRec 4 are shown in the corresponding order. As a byproduct of the results of 
Section |4j the map 7 : S n —> gRec„ induces a lattice structure on gRec n such that 
7 is a surjective lattice homomorphism. 



4. Main theorem 

In this section, we prove our main theorem. 

Theorem 4.1. The restriction of 7 is a bijection from the set of 2- clumped per- 
mutations in S n to the set of generic rectangulations with n rectangles. 



The proof of Theorem |4.1| is accomplished by proving three propositions. 
Proposition 4.2. The map 7 : S n — v gRec n is surjective. 

Proof. Let R! be any diagonal rectangulation and choose an arbitrary wall shuffle 
for each wall of R' . We need to show that there exists x = x\X2 ■ ■ ■ x n £ S n such 
that p(x) = R' and such that each chosen wall shuffle is a subsequence of X1X2 ■ ■ ■ x n . 
That is, we need to show that the rectangles of R' can be ordered consistent with 



the requirements of Proposition 3.2 and with the wall shuffles. 

Suppose, for 1 < i < n, that we have chosen i— 1 rectangles in an order consistent 
with the requirements of Proposition |3.2| and with the wall shuffles. We will show 
that we can choose a rectangle in step i that also satisfies the requirements. Since 
we have chosen consistent with Proposition [3]2] the union T of the i — 1 rectangles 



chosen with the left and bottom edges of S, is a left- and bottom-justified set. 



To satisfy the requirement of Proposition 3.2 in step i, we must chose a rectangle 



whose bottom and left edges are contained in T. To show that we can choose such 
a rectangle consistent with the wall shuffles, we extend an argument from the proof 
of [TH Proposition 6.2]. 
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Figure 5. A figure illustrating the proof of Proposition 4.2 



The top-right boundary of T is a polygonal path from the top-left corner of S to 
the bottom-right corner of S, always moving directly right or directly down. Each 
point where the path turns from moving down to moving right is the bottom-left 
corner of a rectangle of R! that is not contained in T. We index these rectangles 
Ui, . . . , U m from top-left to bottom-right. The left edge of U\ is necessarily con- 
tained in T, or else we were wrong to index it as U\. Thus if U\ fails to have both 
its bottom and left edges in T, then its bottom edge is not contained in T. This 
implies that the left edge of U% is contained in T. We continue until we find the 
first j such that the bottom edge of Uj is contained in T. Since the bottom edge of 
U m is in T, such a j exists. Necessarily, the left edge of Uj is also contained in T. 

We now consider the walls containing the edges of Uj. First, let Wi be the wall 
containing the left edge of Uj . (If j = 1 and the left edge of Uj is in the left edge 
of S, then there is no wall shuffle associated to the left edge of Uj.) Because the 
bottom edge of Uj-i is not contained in T, the top endpoint of Wi is contained in 
the bottom edge of Uj-i, as illustrated in Figure [5j (If j = 1 and the left edge of 
Uj is not in the left edge of S, then the top vertex of W\ is in the top edge of S.) 
We conclude that all rectangles adjacent to and left of Wi are contained in T. Thus 
we can pick Uj in step i consistent with the wall shuffle in W\. 

Second, let W t be the wall containing the top edge of Uj. If W t is also the 
wall containing the bottom edge of Uj-i, then since the bottom edge of t/j_i is 
not contained in T, the top-left corner p of Uj is the top-right corner of another 
rectangle of R! . Since that other rectangle must intersect the diagonal, every point 
on Wt from p rightwards is above the diagonal. Thus Uj-i is the rightmost of the 
rectangles adjacent to and above W t , because otherwise the bottom-right corner 
of Uj-i is the bottom- left corner of a rectangle of R' that doesn't intersect the 
diagonal. Since the left edge of Uj-\ is contained in T, all other rectangles adjacent 
to and above W t are constructed in steps 1 through i — 1. Thus we can pick Uj in 
step i consistent with the wall shuffle in Wt- If Wt is not the wall containing the 
bottom edge of f7j-_i, then the left endpoint of W t is also the top- left corner of Uj. 
(This is the case that is illustrated in Figure [5]) In this case, Uj is leftmost among 
rectangles adjacent to and below Wt, so Uj does not figure in the wall shuffle in Wt- 

We have shown that picking Uj in step i is allowed by Proposition 3.2 and by 
the wall shuffles in the walls Wi and Wt- Let W r be the wall containing the right 
edge of Uj and let Wb be the wall containing the bottom edge of Uj, if these exist. 
If j = m, then there is no wall W r and either there is no wall Wt or Uj is the 
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rightmost rectangle adjacent to and above Wb, so that Uj does not figure in the 
wall shuffle in Wb- Thus if j = m, the rectangle Uj can be picked in step i. If, on 
the other hand, j < m, then Uj can be picked in step i if and only picking it is 
allowed by the wall shuffle in W r and by the wall shuffle in Wb- 

Let W[ be the wall containing the left edge of Uj+% and let W' t be the wall 
containing the top edge of Uj+\. We will prove the following claim: If picking Uj 
in step i is disallowed by the wall shuffle in W r , or if it is disallowed by the wall 
shuffle in Wb, then the left edge of Uj+i is contained in T, and picking Uj+i in step 
i is allowed by the wall shuffle in W[ and by the wall shuffle in W' t . 

First, suppose that picking Uj in step i is disallowed by the wall shuffle in W r . 
If the bottom endpoint of W r is also the bottom-right corner of Uj (as shown in 
Figure [5]), then Uj is the lowest of the rectangles adjacent to and left of W r . This 
would contradict the supposition that picking Uj next is disallowed by the wall 
shuffle in W r , so we conclude that the wall W r continues below Uj. Since the 
bottom edge of Uj is contained in T, it follows that the bottom right corner of Uj 
is the next convex corner of T . In particular, the wall shuffle in W r requires us to 
choose Uj+i before Uj. Since the wall W[ coincides with W r , we know that the wall 
shuffle in W{ does not prevent us from choosing Uj + \ next. Also, we see that the 
left edge of Uj+i is contained in T: Otherwise the right edge of Uj intersects the 
left edge of Uj+i, making Uj+i the topmost of the rectangles adjacent to and right 
of W r (because R' is a diagonal rectangulation) . This contradicts the supposition 
that picking Uj next is disallowed by the wall shuffle in W r . Furthermore, we see 
that the top-left corner of Uj+i is strictly below the convex corner of T separating 
Uj from Uj+\: Otherwise, that convex corner is the corner of four rectangles of 
R' (including Uj and f/j+i). Thus Uj+\ is the leftmost rectangle adjacent to and 
below W' t , so it does not figure in the wall shuffle in W[. 

Next, suppose that picking Uj in step i is disallowed by the wall shuffle in Wb- Let 
V be leftmost among rectangles adjacent to and below Wb that are not contained 
in T. The rectangle V is also shown in Figure [5] Since picking Uj in step i is 
disallowed by the wall shuffle in Wb, the rectangle V exists and comes before Uj in 
the wall shuffle for Wb- The top endpoint of the wall W{ is the top- left corner of 
V, and in particular is contained in T. Thus all of the rectangles adjacent to and 
left of W( are in T, so that choosing f/j+i in step i is allowed by the wall shuffle 
in Wi. If V = Uj + i, then W[ = Wb, and we already know that the wall shuffle in 
Wb requires us to pick V next. If V is not Uj+i, then the left endpoint of W[ is 
contained in T, so Uj+i is the leftmost rectangle adjacent to and below W[, and 
thus Uj+i does not figure in the wall shuffle in W[. In either case, the left edge of 
Uj+i is contained in T, and we have proved the claim. 

If the bottom edge of Uj + \ is contained in T, then the claim implies that picking 



Uj+i in step i is allowed by Proposition 3.2 and by the wall shuffles in the walls W[ 



and W[. We can thus argue for Uj + \ just as we have argued above for Uj. If the 
bottom edge of Uj+\ is not contained in T, then we find the first k > j such that 
the bottom edge of Uk is contained in T, and start over as above, replacing Uj with 
Uk- Eventually, we will find a rectangle that can be picked, because the bottom 
edge of U m is contained in T and because, as mentioned above, wall shuffles in the 
walls below U m and to the right of U m will never prevent its being picked. □ 



Proposition 4.3. Suppose x<y in the weak order, and let e and a be the adjacent 
entries that are swapped to convert y to x, with a < e. Then 7(2) = 7(3/) if and 
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Figure 6. Figures illustrating the proof of Proposition 



4.3 



only if there are entries b, c, and d in y, with a<b<c<d<e, such that b and d 
are on the same side of ea, while c is on the other side of ea. 

Proof. Both conditions in the proposition imply that p(x) = p(y), by the definition 
of 7 and by Proposition 3.2 Throughout the proof, let R' be the diagonal rectan- 
gulation p{x) = p{y). We claim that j(x) — "f(y) if and only if U a and U e are not 
adjacent to any common wall of R' . Indeed, if U a and U e are not adjacent to any 
common wall of R' , then x and y must define the same wall shuffles on the walls of 
R' , so 7(x) = 7(2/). Conversely, suppose U a and U e are adjacent to a common wall 
W of R'. The assumption that p(x) = p(y) rules out the possibility that U a and U e 
are both on the same side of W, so U a and U e are on opposite sides of W. Since U a 
is chosen immediately before U e when R' is constructed as p(x) but immediately 
after U e when constructing R' as p(y), we see that a and e are both entries in aw- 
That is, if W is vertical, then neither U a nor U e is the bottom-most rectangle ad- 
jacent to W on the left, and neither U a nor U e is the topmost rectangle adjacent 
to W on the right. Similarly, if W is horizontal, neither of the two rectangles are 
the leftmost rectangle below W nor the rightmost rectangle above W . We conclude 
that j(x) ^ j(y) and we have proved the claim. 

Suppose there are entries b, c, and d in y, with a<b<c<d<e, such that b 
and d precede ea, but c follows ea. Let T be the union of the left and bottom edges 
of S with the rectangles chosen before U a and U e when R' is constructed as p(x) or 
p(y). In the construction of R' as p(x), the rectangle U a is chosen next, but in the 
construction of R' as p(y), the rectangle U e is chosen next. Thus both T U U a and 
T L)U e are bottom- and left-justified sets. Therefore, every point of U e is strictly 
below and strictly to the right of every point of U a - Sec Figure [6ja, ignoring, for 
now, the labels p a and p e . In particular, U a and U e are not adjacent to a common 
wall of R' . By the claim, 7(2;) = 7(2/). 

Similarly, if there are entries 6, c, and d in y, with a<b<c<d<e, such that 
c precedes ea, but b and d follow ea, we see that every point of U e is strictly below 
and strictly to the right of every point of U a . See Figure |6]b, ignoring the labels p, 
p a , and p e . In particular, U a and U e are not adjacent to a common wall of R' , so 
70) = 7(2/)- 

Conversely, suppose 7(2;) = 7(1/). Let T be as above. For every concave corner p 
of the top-right boundary of T, there is a rectangle of U whose bottom-left corner 
is p. Let p a be the bottom-left corner of U a and let p e be the bottom-left corner 
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of U e . Both p a and p e are concave corners of the boundary of T. There are two 
possibilities: The first is that, looking from top-left to bottom-right at the concave 
corners of the boundary of T, there is some concave corner between p a and p e . In 
this case, there exist entries b, c, and d in y, with a<b<c<d<e, such that 
b and d are before ea, while c is after ea, as illustrated by Figure [6ja. The second 
possibility is that, from top-left to bottom-right, there are no other concave corners 
between p a and p e . Thus there is a single convex corner p of T between p a and p e . 
Since R' equals both p(x) and p(y), both U a and U e have their bottom and left 
edges contained in T. If p is the bottom- right corner of U a , or if p is the top- right 
corner of U e , then U a and U e are adjacent to a common wall. By the claim, this is 
a contradiction to the supposition that 7(2) = 7(2/)- Thus there exist entries 0, c, 
and d in y, with a<b<c<d<e, such that c is before ea, while b and d are after 
ea, as illustrated in Figure [6|b. □ 



Proposition |4.2| asserts that every fiber of 7 is nonempty. The following proposi- 
tion characterizes the fibers more exactly, and completes the proof of Theorem |4.1 



Proposition 4.4. Each fiber 0/7 is aT -class. In particular, each fiber 0/7 contains 
a unique 2- clumped permutation. 



Proof. By Proposition 2.2|2 and Proposition 4.3 the fibers of 7 are unions of V 



classes. Suppose x and y are distinct permutations with 7(2) = j(y). Then p{x) 
p(y) and x and y are consistent with the same set of wall shuffles. We will show 
that x and y are congruent modulo T. Let i be the smallest index such that Xi =/= yi- 
We argue by induction on n — i. 

There is some k > i such that yj~ = Xi. Since 7(2) = 7(y), either the rectangle 
numbered Xi or the sequence of rectangles numbered yiyi+x • ■ ■ yk can be chosen 



next, consistent with the requirements of Proposition 3.2 and with the wall shuffles. 
We conclude that the entry yk = Xi does not participate in any wall shuffles with 
any of the entries y^+i • • • y k -i- 

Consider the sequence of permutations starting with y and moving the entry yk 
to the left one place at a time, without changing the relative positions of the other 
entries, with the final entry y' in the sequence having y k in position i. Then 7 
is constant on the sequence. Since each pair of adjacent permutations in the se- 
quence is a covering pair in the weak order, each pair is related as described in 
Proposition |4.3| But then Proposition |2.2|2| says that the entire sequence is con- 
tained in one T-class, so that in particular y' and y are congruent modulo T. Since 
7(2) = 7(2/') and x and y' agree in positions 1 through i, by induction we conclude 
that x and y' are congruent modulo T. Thus x and y are congruent modulo T. 

We have shown that each fiber of 7 is a T-class. The second assertion of the 
proposition follows by Proposition |2.2|l| □ 

5. Remarks on enumeration 
A pleasant formula was obtained in [7] for the number of Baxter permutations 



B{n) 



"rn-rrtfrocrx::; 



fc=i 

This formula applies to twisted Baxter (i.e. 1-clumped) permutations and to diago- 
nal rectangulations as well. In this section, we make several remarks on the problem 
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of enumerating generic rectangulations or 2-clumped permutations. In particular, 
we give some indications that the enumeration of 2-clumped permutations will be 
harder than the enumeration of 1-clumped permutations. 

Remark 5.1. One way to enumerate generic rectangulations is by specializing a 
formula of Conant and Michaels [5]. This formula is a recursion, with signs, counting 
rectangulations according to the number of crosses. Thanks to Jim Conant for 
providing the results of his recursive calculations which verify and extend Table [T] 

Remark 5.2. Another approach to enumerating 2-clumped permutations is to apply 
the key idea from [7] . This approach appears not to lead to a formula for the number 
of generic rectangulations, but is useful computationally, as we now explain. 

Suppose x G G n . For each entry a in x, let /3(a) = {b G [a + 1, n] : b is before a}. 
Then n + 1 can be placed before a in x to obtain another 2-clumped permutation if 
and only if one of the following holds: [3(a) — 0, /3(a) = [a + 1, n], /3(a) = [a + 1, c] 
for some c with a + 1 < c < n, or /3(a) = [d, n] for some d with a + 1 < d < n. 
Notice that if a satisfies none of these requirements, then even after n + 1 is inserted 
elsewhere to obtain a permutation x' G G„+i, the entry a in x' still satisfies none 
of the requirements. Notice also that n + 1 can be inserted after all of the entries 
of x to obtain a permutation in G n +i. 

Accordingly, we encode a 2-clumped permutation by a string of letters as follows. 
Read through the elements of x from left to right, and for each element a, write a 
letter in the string as follows: 



n 


(for 


"null" or "n") 


if a = n. 


Assume a ^ n in the following cases. 


e 


(for 


"empty" ) 


if /3(a) = 


0. 


f 


(for 


"full") 


if /3(a) = 


[a + 1, n]. 


1 


(for 


"lower" ) 


if /3(a) = 


[a + 1 , c] for some c with a + 1 < c < n 


u 


(for 


"upper" ) 


if /3(a) = 


[d, n] for some d with a + 1 < d < n. 



If none of these apply, then write nothing. 

For example, for each of the permutations 2413, 4213, 3124 and 3142, the sym- 
bol 5 can be inserted anywhere except before the symbol 1. The sequences of 
letters for these permutations are respectively en • f, nu • f , e • In, and e • nf, with a 
dot "•" indicating an entry in the permutation that does not produce a letter. The 
respective strings are enf, nuf, eln, and again enf. 

If we place the symbol n + 1 before a in x or if we place n + 1 after all entries 
of x, we can construct the string of letters corresponding to the new permutation 
x' by the following procedure. Insert the letter n in the string before the letter 
corresponding to a or at the end of the string and alter letters before the insertion 
according to the following rule: n becomes e, e is unchanged, f becomes 1, 1 is 
unchanged, and u is deleted. Alter letters occurring after the insertion as follows: 
n becomes f, e becomes u, f is unchanged, 1 is deleted, and u is unchanged. 

Now we can dispense with permutations entirely and simply insert letters into 
strings, counting the resulting strings by multiplicities. We start with the string 
n, encoding the permutation 1 G G\. Inserting before or after the one letter in 
the string, we obtain en and nf, corresponding to the permutations G2 = {12, 21}. 
Inserting into these two strings, we obtain the strings een (for 123), enf twice (for 
132 and 231), nuf (for 312), eln (for 213), and nff (for 321). In the next round of 
insertions, deletions of letters come into play, so that for example, inserting n after 
the e in eln, we obtain enf. This corresponds to inserting 4 after 2 in 213 to obtain 
2413. In all, there are 15 strings which represent the 24 permutations in G4. 
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The values shown in Table [T] are the results of a simple computer program that 
generates all strings and keeps track of multiplicities. 

In contrast, representing 1-clumped permutations (i.e. twisted Baxter permuta- 
tions) by strings leads to an enumeration formula. In this case the locations where 
n + 1 can be inserted are the locations labeled n, e, or f, with the same defini- 
tions as above. When n is inserted into the string, the remainder of the string 
is altered as follows: Before the insertion, n becomes e, e is unchanged, and f is 
deleted. After the insertion, n becomes f, e is deleted, and f is unchanged. All of 
the strings are of the form e l nf J , for i,j > and i+j < n— 1. Define G(n,i,j) 
to be the multiplicity of the string e l nf J for 1-clumped permutations in S n . Up 
to reindexing in n, the numbers G(n,i,j) coincide with the numbers T n (i,j) in [7], 
and the obvious recurrence on G(n, coincides with the recurrence on T n (i,j). 
This recurrence can be solved as in [7j , or by the generating function method of [B] . 
In particular, the generating tree for the twisted Baxter permutations is isomor- 
phic to the generating tree for Baxter permutations. Indeed, the original proof [18] 
that twisted Baxter permutations biject with Baxter permutations proceeded by 
establishing this isomorphism of generating trees. 

Remark 5.3. Mallows 1(5] gave a combinatorial interpretation for the terms in 
formula for B{n) by pointing out that the term indexed by k counts Baxter permu- 
tations with k ascents (or rises). There are two dual ways to define ascents: We 
will say that a right ascent is a pair of adjacent entries such that the left entry in 
the pair is smaller than the right entry in the pair. A left ascent is a pair of entries 
i and i — 1 with i — 1 appearing before i in the permutation. We can similarly 
define right descents (left entry in the pair larger) and left descents (i—l appearing 
after i). Recall that the Baxter permutation are the permutations avoiding 3-14-2 
and 2-41-3. It is easy to see that a given permutation is a Baxter permutation if 
and only if its inverse is a Baxter permutation. (See e.g. [T51 Corollary 4.2].) Thus, 
when counting Baxter permutations according to the number of ascents, it does not 
matter whether we use right ascents or left ascents. Furthermore, it is immediate 
that a permutation is a Baxter permutation if and only if its reverse permutation 
is also a Baxter permutation. Thus the formula for Baxter permutations with a 
fixed number of ascents is the same as the formula for Baxter permutations with a 
fixed number of descents. It is easy to see that the number of ascents in a permuta- 
tion x equals the number of vertical walls in the diagonal rect angulation p(x). Thus 
the formula for B(n) counts diagonal rectangulations according to the number of 
vertical walls. 

The number of left ascents of x also equals the number of vertical walls in the 
generic rectangulation j(x) and the number of left descents of x equals the num- 
ber of horizontal walls. Thus by the symmetry of the rectangulations, counting 
2-clumped permutations by left descents is equivalent to counting 2-clumped per- 
mutations by left ascents. However, the inverse of a 2-clumped permutation is not 
necessarily a 2-clumped permutation, so it matters whether we take the left or right 
definitions of descents or ascents. Thus there are at least three reasonable statistics 
by which to count: left ascents/descents, right ascents, or right descents. Compu- 
tations show that these three statistics are distributed differently, and suggest the 
following conjecture: 

Conjecture 5.4. Fix k > 0. Then for n > 1, the number of 2-clumped permutations 
in S n with exactly d right descents is a polynomial p^ (n) of degree 3d and leading 
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coefficient 



fJ i i{i+l){i + 2) d\{d+l)\{d + 2)V 

The polynomial pk(n) must have factors (n — l)(n — 2) • • • (n — d), so the point 
is to determine the polynomial pk{n) of degree 2d that results when these factors 
are taken out. The first few polynomials appear to be pa(n) = 1, 

p^n) = (n 2 -2n + 3)/3, 

p 2 (n) = (5n 4 - 36n 3 + U2n 2 - 279n + 270)/180, and 

p 3 (n) = (Un 6 - 213n 5 + 1688n 4 - 8361n 3 + 26000n 2 - 46884n + 37800)/15120. 

It should be emphasized that the point of the conjecture is to find a formula enu- 
merating all 2-clumped permutations. The conjecture can be proved for some small 
values of k, and proofs for additional values of k are of interest only to the extent 
to which they lead to a conjecture on the general form of pk{n). 

The other two statistics (left ascents/descents and right ascents) do not lead 
to polynomial formulas. In particular, counting 2-clumped permutations by left 
ascents, or equivalently counting generic rectangulations by the number of vertical 
walls, appears to be hard. 
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